Expressive Models and Comprehensive Benchmark for 2D Human Pose Estimation

In this work we consider the challenging task of articulated human pose estimation in monocular images. Most of current methods in this area [4, 8, 16, 14] are based on the pictorial structures model (PS) and are composed of unary terms modelling body part appearance and pairwise terms between adjacent body parts and/or joints capturing their preferred spatial arrangement. In this work we advance the state of the art in articulated human pose estimation in three ways. First, we argue that modeling part dependencies between non-adjacent body parts is important for effective pose estimation (cf. Fig. 1). We propose a model [10] that incorporates higher order information between body parts by defining a conditional model in which all parts are a-priori connected, but which becomes a tractable PS model once the mid-level features are observed. This allows to effectively model dependencies between non-adjacent parts and retains an exact and efficient inference procedure in a tree-based model. Second, we explore various types of appearance representations with the aim to improve the body part hypotheses [11]. We argue that in order to obtain effective part detectors it is necessary to leverage both the pose specific appearance of body parts and the joint appearance of part constellations. We show that the proposed appearance representations are complementary and a combination of the best performing appearance model paired with a flexible image-conditioned spatial model achieves the best result. Third, we introduce a novel benchmark “MPII Human Pose” [3] that makes a significant advance in terms of diversity and difficulty, a contribution that we feel is required for future developments in human body models. This comprehensive dataset was collected using an established taxonomy of over 800 human activities. The collected images cover a wider variety of human activities than previous datasets including various recreational, occupational, and householding activities. People are captured from a wider range of viewpoints. In addition we provide a rich set of labels including positions of body joints, full 3D torso and head orientation, occlusion labels for joints and body parts, and activity labels. With these annotations we perform a detailed analysis [3, 12] of the leading 2D human pose estimation and activity recognition methods to understand success and failure cases for established models.